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Abstract —Mining the underlying patterns in gigantic and complex data is of great importance to data analysts. In this paper, 
we propose a motion pattern approach to mine frequent behaviors in trajectory data. Motion patterns, defined by a set of highly 
similar flow vector groups in a spatial locality, have been shown to be very effective in extracting dominant motion behaviors in 
video sequences. Inspired by applications and properties of motion patterns, we have designed a framework that successfully 
solves the general task of trajectory clustering. Our proposed algorithm consists of four phases: flow vector computation, motion 
component extraction, motion component’s reachability set creation, and motion pattern formation. For the first phase, we break 
down trajectories into flow vectors that indicate instantaneous movements. In the second phase, via a Kmeans clustering 
approach, we create motion components by clustering the flow vectors with respect to their location and velocity. Next, we 
create motion components’ reachability set in terms of spatial proximity and motion similarity. Finally, for the fourth phase, we 
cluster motion components using agglomerative clustering with the weighted Jaccard distance between the motion components’ 
signatures, a set created using path reachability. We have evaluated the effectiveness of our proposed method in an extensive 
set of experiments on diverse datasets. Further, we have shown how our proposed method handles difficulties in the general 
task of trajectory clustering that challenge the existing state-of-the-art methods. 

Index Terms —Trajectory, Flow vector, Motion component, Reachability, Motion Pattern. 
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1 Introduction 

ecently, there has been a great amount of research 
in mining frequent behaviors from trajectory data. A 
trajectory is an N dimensional path that does not necessarily 
correspond to a physical object, but may correspond to 
the time evolution of an N dimensional feature vector. 
In this paper, we propose a novel approach, inspired by 
the concept of motion patterns, for extracting dominant 
trajectory behaviors; while our method will be described for 
the case of 2 dimensional trajectories, it can be extended to 
higher dimensions for further applications. Seeking to find 
common behaviors in trajectory data, some methods group 
whole trajectories into clusters while others attempt to mine 
regional patterns that trajectories follow during part of their 
evolutions. Our proposed method aligns more closely to the 
latter category. In this section, we begin with introducing a 
subset of very rich literature in trajectory clustering and 
try to keep a balance between the recent methods and 
popular ones. Due to the existence of a large number of 
methods in the literature and the space limitation, selecting 
a subset of the literature is inevitable. Next, we explain how 
methods performing scene activity understanding in video 
sequences can be seen as trajectory clustering tools. And 
finally, we intuitively introduce our motion pattern approach 
as a general tool for understanding trajectory behaviors. 

Many of the existing trajectory clustering methods ap- 
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proach the problem by first defining a similarity function 
for trajectories and then using one of the well-established 
clustering procedures. For instance, [1] compared clustering 
results obtained by using the trajectory similarity measures 
Dynamic Time Warping (DTW), Longest Common Subse¬ 
quence (LCSS), and modified Hausdorff distance in combi¬ 
nation with agglomerative and spectral clustering. Fu et.al. 
[2] used the average distance between corresponding trajec¬ 
tory points as the similarity measure, which required pre¬ 
processing and resampling of trajectories. The clustering 
was later applied in two steps, with the first step producing 
clusters corresponding to the larger dominant paths, and 
the second one subsequently refining results obtained by 
the first one. Morris and Trivedi [3] surveyed performance 
of a wide variety of distance measures and clustering 
algorithms on several datasets with varying characteristics. 
Specifically, they compared trajectories using [4], PC A 
[5], DTW [< ], LCSS [7], [ ], and modified Hausdorff 
[S] distance and then obtained clusters using direct [10], 
divisive [11], agglomerative [7], hybrid [1 ], graph [13] and 
spectral clustering techniques [4]. Ferreira et.al. [14] have 
proposed Vector field Kmeans that treats the trajectories as 
a whole and attempts to follow an iterative model similar 
in form to Kmeans clustering. In this approach, clusters 
are subsets of the trajectories. First, the trajectories are 
partitioned randomly into K clusters. Then, for each cluster, 
the trajectories in that cluster are used to form one of K 
vector fields. Further, clusters are updated by assigning each 
trajectory to the vector field that it fits best. These two 
steps are repeated in an iterative fashion until a convergence 
criteria is met. DivCluST proposed by Wu et.al. [1: ] is a 
partition based approach that clusters trajectories according 
to their speed in addition to their shape and position. First, 
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it partitions the trajectories into representative segments by 
combining adjacent trajectory segments that have similar 
direction and speed. Then, a method similar to Kmeans 
is used to find cluster centers. The clusters are initialized 
as random subsets of the representative segments. Next, 
the representative segments of each cluster are averaged to 
form a cluster center. Finally, as in Kmeans, each of the 
representative segments is associated with a cluster center, 
then each cluster center is updated to be the average of 
the cluster’s representative segments. This process will be 
repeated until a convergence criteria is met. Ulm et.al. 
[16] have proposed a trajectory clustering approach that 
is distinctive in its online clustering ability. It does not 
explicitly treat the trajectories as a whole, but in practice, it 
behaves as if it does. They defined a cluster as a vector field 
defined on a subset of the spatial range. When a trajectory 
is received, it is converted into a new vector field. If at 
any time two vector fields are similar, they are merged 
together into one. This method uses the entire trajectory to 
make a vector field and merging process largely conserves 
the shape of both fields that are being merged. Thus, the 
algorithm in practice behaves similarly to those approaches 
that treat the trajectories as a whole. 

Giannotti et.al. [17] proposed a very different way of 
treating trajectories where their density is used to form 
frequently visited rectangular regions. Then, each of the tra¬ 
jectories is represented as a sequence of visited regions with 
the transition times. They mined common subsequences 
with similar transition times that represent commonly taken 
paths similar in interpretation to trajectory clusters. It is 
worth mentioning that because of the exact phrasing of 
the problem, this approach gives very redundant clusters in 
practice. Lee et.al. [18] observed that clustering trajectories 
as a whole fails to reveal portions of trajectories that 
exhibit a common behavior. To address this, they proposed 
TRACLUS, an algorithm that first partitions trajectories 
into a set of line segments and then groups similar segments 
into clusters using a density-based clustering algorithm for 
line segments (largely analogous to DBSCAN). This work 
has been further extended in [19], which proposed a feature 
generation framework, TraClass, that uses a combination of 
region-based and trajectory-based clusterings. Specifically, 
TraClass finds homogeneous regions where trajectories of 
a certain class are predominant, then uses a class-conscious 
adaptation of TRACLUS to group trajectory partitions into 
clusters that have high discriminative power. After gener¬ 
ating features in this manner, TraClass maps trajectories 
into a feature vector, where each entry corresponds to a 
region-based or a trajectory-based cluster. 

1.1 Scene Activity Understanding Methods 

Given the decreasing cost of collecting data and the 
great importance of surveillance videos for providing se¬ 
curity and monitoring ongoing activities, many researchers, 
mostly in the computer vision community, dedicated major 
efforts to develop automatic scene modeling and intelligent 
activity understanding systems. These research works aim 
to learn frequent movement behaviors and activity profiles 
in videos. Using that, they can also detect abnormal be¬ 


haviors and in some cases improve object tracking perfor¬ 
mance. Given a video sequence, some of these methods 
obtain trajectories associated with the moving objects in 
the scene, while others, due to the challenges such as 
highly dense scenes, frequent moving object occlusions or 
poor object tracking performance, either compute short¬ 
term trajectories (tracklets) or compute sparse [20] or dense 
optical flows [2 ] in the feature extraction phase. Then, 
different kinds of algorithms are employed to infer frequent 
motion behaviors taking place in the video sequences from 
these extracted features. In this paper we will refer to these 
frequent motion behaviors as motion patterns. Semantically, 
motion patterns are very similar to the regional patterns 
mined by those groups of trajectory clustering methods 
that seek common behaviors in subsegments of trajectories. 
Therefore, we claim that a motion pattern approach can be 
employed to solve the general task of trajectory clustering. 

Junejo et.al. [22] cluster similar trajectories by per¬ 
forming a min-cut graph clustering algorithm recursively. 
Each node in the graph represents a trajectory while the 
weights of the edges between nodes are determined by 
the Hausdorff distance between corresponding trajectories. 
Modeled paths are later used to detect unusual trajecto¬ 
ries based on spatial velocity and curvature features. [23] 
computes instantaneous flow vectors that include location 
and velocity information from either optical flows or long¬ 
term/short-term trajectories. Given the location of a point 
and it’s velocity (based on flow vectors), a kernel based 
estimation similar to mean shift approach [24] is employed 
to generate the corresponding velocity in the next step. 
The next location of each point can be estimated using 
its current location and computed velocity. Performing 
this procedure recursively will generate the sink path that 
estimates the movement path of the initial point. Next, 
obtained sinks are clustered based on the similarity of 
their location and directions in addition to the Hausdorff 
distance between their sink paths. These clusters represent 
the frequent motion behaviors. Basharat et.al. [25] have 
proposed a framework that does not explicitly generate 
motion patterns; however, the per-pixel learned multivariate 
probability density function can be used to understand the 
frequent motion behaviors in the scene. Given a set of 
trajectories obtained from moving objects, transition vectors 
that represent size of the moving object, transition time 
and location of the object after the transition are computed. 
Then, for any given location in the scene, transition vectors 
that pass through that location contribute to a multivariate 
Gaussian Mixture Model (GMM) that models the pixel- 
level pdf. Given such a model, most likely paths can be 
generated by initializing the starting points in the same 
fashion as [26] or by taking a Markov Chain Monte Carlo 
(MCMC) sampling based scheme such as [27]. Hu et.al. 
[28] compute flow vectors by extracting sparse or dense 
optical flow from video sequences. Then, they create a 
neighborhood directional graph that indicates the distance 
between flow vector pairs. Finally, a hierarchical agglomer- 
ative clustering algorithm generates motion patterns as the 
graph clusters of the neighborhood graph. Saleemi et.al. 
[2 ] have proposed a probabilistic framework to model 
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scene dynamics. Given the long-term/short-term trajectories 
of moving objects in a video sequence obtained from an 
object tracking algorithm, a 5 dimensional feature space 
is created where each feature point represents the location 
information of initial and final stages of a moving object 
and the duration of the time interval for such a transition. 
Then, motion patterns , in the form of a multivariate pdf 
of spatiotemporal parameters, will be learned using kernel 
density estimation. Finally, a Markov Chain Monte Carlo 
(MCMC) sampling based scheme is utilized to sample the 
pdf. For any starting location as the initial state, MCMC 
sampling based scheme generates the probable random 
walks that are most likely to happen in the scene in a 
progressive procedure. These likely paths express the most 
probable paths that trajectories have taken in the training 
stage. Therefore, those are semantically similar to motion 
patterns. Lasdas et.al. [ 26 ] present a method to extract 
dominant dynamic properties in crowded scenes. Their pro¬ 
posed pipeline begins with extracting fixed length tracklets 
using KLT tracker [ 29 ]. Then, a set of validation tests 
and pre-processing steps refine the tracklet set. In order to 
have a more robust representation, a grid of equally spaced 
points is overlaid on the scene. The pipeline continues 
with clustering tracklets in the neighborhood of each of 
these points using a mean shift clustering algorithm with 
respect to the direction of the tracklets. Next, mean flow 
vectors that are assigned to each grid point are modeled 
as a Gaussian Process (GP) using [ 30 ]. Finally, given a 
starting point and the mixture of GP regression models, a 
sequential sampling scheme generates the trajectories that 
follow the most frequently observed patterns in the training 
phase. These generated paths are supposed to be the most 
probable paths that might originate from the starting point 
and are semantically similar to the motion patterns. 

Zhao and Medioni [3 ] proposed a framework to infer 
motion patterns in videos in an unsupervised learning 
fashion. Given a video with moving objects, they extract 
foreground motion blobs (connected pixels that belong to 
a moving object) using Robust Alignment by Sparse and 
Low Rank Decomposition (RASL) [ 32 ]. Performing local 
associations using [ 33 ] and [ 34 ], they obtain the initial 
tracklets. Here the tracklets are sequences of ordered spatial 
coordinates of motion blobs. Then, using a 2 dimensional 
Tensor Voting algorithm [ 35 ], they compute the tangent 
direction of every tracklet point. The direction informa¬ 
tion provided after 2 dimensional Tensor Voting is less 
noisy and more consistent than transition vectors from one 
observation to the next one. Further, a similarity graph 
of tracklets based on multiple kernel types is generated 
and, employing a graph spectral clustering approach, the 
nonlinear manifold of tracklets is grouped into segments. 
These segments indicate local motion patterns; while [31] 
focuses on global motion patterns. Therefore, a kernel 
density estimation algorithm is utilized to propagate the 
local motion pattern information to all the pixels in the 
scene and form global motion patterns. As the last piece 
of work to cover in this section, we briefly explain the 
approach that [ 36 ] took to provide a scene understanding 
framework, a framework that we believe is capable of 


solving the trajectory clustering task. The pipeline begins 
with dividing a video into disjoint video clips and extracting 
dense optical flow from them. Then, each video clip is 
represented as a 4 dimensional feature space that contains 
location, magnitude and orientations of the instantaneous 
optical flow vectors belonging to that particular video clip. 
Next, each of the 4 dimensional feature spaces are modeled 
as a mixture of Gaussians where Kmeans clustering is 
employed to initialize the model parameters. The pipeline 
continues with generating a graph that treats each Gaussian 
component as a node where the edge between two nodes 
is determined by their reachability from each other and 
associated temporal proximity. To clarify, the reachability 
is defined as the probability of observing a component 
in the neighborhood of the other one in a specific time 
interval. Finally, through a connected component analysis 
on the aforementioned graph, motion patterns are obtained. 
Since similar motion patterns might occur at different time 
stamps, Kullback-Leibler (KL) divergence is employed to 
merge similar motion patterns happening at different times. 

1.2 Proposed Approach 

We explained the trajectory clustering problem and gave 
a brief overview of a group of methods that seek to find 
regional patterns in trajectory data rather than clustering 
whole trajectories. Then we showed that the motion pattern 
inference methods trying to perform scene understanding 
in video sequences are semantically generating the same 
output as the aforementioned trajectory clustering meth¬ 
ods. Therefore, we claim that a motion pattern approach 
can be employed to solve the general task of trajectory 
clustering. In this paper, we propose a novel framework 
that is designed to overcome the challenges provided by 
the trajectory clustering task. Given a set of trajectories, 
we break them down into sets of flow vectors and generate 
a 4 dimensional feature space. Each point in this feature 
space represents location and velocity of a flow vector. 
Then, using Kmeans clustering, we represent the feature 
space in a less noisy and compact form. Clusters obtained 
from Kmeans clustering are called motion components. 
Next, we define the reachability set in which a pair of 
two motion components exist if they are reachable from 
each other. The reachability set maintains local similarities 
between motion components and cannot evaluate motion 
component’s contribution in generating the global trajectory 
behaviors. Hence, we define the signature which represents 
global behavior of a motion component. Finally, a weighted 
Jaccard distance between motion components’ signatures is 
used as the distance metric in an agglomerative clustering 
scheme to find motion patterns in trajectory data. These 
motion patterns illustrate regional behaviors of trajectories 
and are semantically similar to the trajectory clusters of 
those methods which break trajectories into segments and 
seek for regional similarities among them. The main con¬ 
tributions of our proposed method are as follows: 

• We propose a motion pattern approach to mine tra¬ 
jectory clusters of arbitrary shapes, simultaneously in 
dense and sparse regions of data. Generally, trajec¬ 
tory clustering methods either mine clusters in dense 
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regions and lose clusters in regions where trajecto¬ 
ries are spread sparsely or extract clusters in sparse 
regions while generating redundant clusters in dense 
regions. 

• The proposed approach is capable of recovering the 
whole trajectory cluster as a single motion pattern 
even in cases of highly curved underlying behaviors. 
Most of the trajectory clustering methods break down 
the whole cluster into segments as they fail to establish 
the connection between the segments when trajectories 
happen to have highly curved shapes. 

• By defining the signatures of motion components, we 
combine the local and global contributions of motion 
components such that it helps us distinguish between 
trajectory clusters that are identical in some localities 
but are different in the global scale. This strategy 
is useful for mining trajectory clusters that merge 
together or split into multiple ones. 

• Generally, trajectory clustering methods cluster sim¬ 
ilar trajectories that are spread over a wide region 
into multiple clusters as model parameters are set 
globally. However, our proposed approach generates 
only a single cluster in these cases as the capability 
of allowing semi-lateral movement is embedded in its 
framework. 

• The proposed method is able to correctly cluster two 
spatially overlapping groups of trajectories that differ 
only in the movement direction. 

The remainder of this paper is organized as follows. 
Section 2 presents the problem definition and formulation 
details of our proposed method. Then, the detailed descrip¬ 
tion of the experiments is given in Section 3, followed by 
the implementation details in Section 4. In Section 5, we 
discuss how our motion pattern approach can be used to 
explore temporal changes in frequent trends in the data with 
applications in various domains. Finally, we conclude the 
paper in Section 6. 

2 Methodology 

In this section, we will introduce our algorithm for mining 
frequent trajectory behaviors. Each block in our pipeline 
will be explained in detail while associated parameters 
will be discussed separately in the implementation details 
(Section 4). 


2.1 Flow Vector Computation 

Given a dataset of N trajectories, we represent its j th 
trajectory with length Lj as an ordered sequence of points 
where denotes the location of j th 

trajectory at time t\ and i = 1, 2,..., Lj. Then its cor¬ 
responding flow vector is a set of 4 dimensional vectors 

x i = (■ x i , Vi, Xi+i~x \, yl+i-yl ) = (xj,yl,ul,vj) where 
i = 1, 2,..., Lj — 1. The first and second dimensions of the 
flow vector indicate its location while the third and forth 
dimensions indicate its velocity. After obtaining flow vec¬ 
tors from all trajectories in the dataset, we will have a vast 
set of 4 dimensional flow vectors (X = (x,y,u,v)) that 
indicate magnitude (s/u 2 + v 2 ) and direction (£an -1 (^)) 


for the movement of each trajectory at all locations in 
the 2 dimensional surface (x and y). This data is noisy 
and we cannot directly use it to understand the behavioral 
movement of trajectories. Therefore, a Kmeans clustering 
algorithm is employed to group it into segments of spatially 
proximal flow vectors that have similar velocity properties. 
Such segmentation of the flow vector set provides us homo¬ 
geneous regional segments that are representative and less 
noisy. We should note that, since in many cases trajectories 
are defined as sequences of points and the actual time 
stamp for each point is not available, only the order of 
the sequence is used in computing the velocity. 

2.2 Motion Component Extraction 

In order to generate the intermediate representation of the 
data that is less noisy and more suitable for the process 
of mining motion patterns, we cluster the flow vector set 
using Kmeans clustering where the number of clusters is 
set to K. Obtained clusters are the motion components. The 
distance measure between p th flow vector of m th trajectory 
and q th flow vector of n th trajectory used to create motion 
components is shown in Equation 1. Note that (3 defines 
the weight of velocity similarity versus spatial proximity, 
of flow vectors in formation of motion components and will 
be discussed in implementation details section. 

Z)(X™, X q ) 2 = (x™ - x ”) 2 + « - y n q ) 2 + 

P(u™-u n q ) 2 + p{v™-v n q ) 2 ( 1 ) 

As the output of this step, we will have K motion compo¬ 
nents that are represented by their 4 dimensional means 
as Mi ^ (^f , p}(, pj-, pf) where i = 1,2, ...,K. We 
also denote the vector that spatially connects m th motion 
component to the n th one by p mn = [(y* - y^),(y y n - 
ptfn)]. This definition will be used in the following section 
to determine the reachability of two motion components. 
Given a sample trajectory set, Figure 1 illustrates the steps 
which our method takes to extract motion components. We 
should note that unlike [ 25 ] and [ 27 ] that aim at learning 
a mixture of probability density functions or a unified 
pdf, respectively, to model the underlying dynamics of the 
entire scene, we cluster the flow vectors only to generate 
intermediate local representations. 

2.3 Motion Component Reachability Set Creation 

We now look for reachability between the motion com¬ 
ponents. Mathematically, reachability is an asymmetric 
relation between motion components. Intuitively, motion 
component A is reachable from motion component B if 
a particle traveling with the initial motion and position 
prescribed by A could reasonably be expected to proceed 
to travel to B. By reachability, we mean direct reachability 
that does not require any intermediate motion components 
between A and B. However, this intuitive concept must 
be formalized and defined mathematically in order to be 
used. We formalize reachability as the conjunction of three 
conditions: 1) The motion components A and B must be 
spatially located close together, 2) The direction of flow 
vector of A should be similar to the direction of the vector 
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Figure 1 . Given a trajectory set like 1(a), we begin with computing flow vectors and obtain 1(b). Then, a Kmeans clustering algorithm is 
employed to cluster the 4 dimensional flow vector set into homogeneous groups. To have a more clear visualization, we first show those flow 
vectors that are spatially proximal in different colors in 1 (c). Since the Kmeans clustering is in 4 dimensions, only those flow vectors that are 
similar in velocity in addition to being spatially proximal will get grouped together. Figure 1(d) shows flow vectors that are spatially proximal 
but differ in velocity properties (magnitude and orientation). The 4 dimensional Kmeans clustering provides clusters like those illustrated in 
1 (e) where flow vectors that are spatially proximal (shown in 1 (c)) and have similar velocity (shown in 1 (d)) are grouped together. 


Pab , 3) Direction of flow vectors for A and B should 
be similar. Intuitively, the second condition enforces flow 
vector of A to be aligned with shortest spatial path from A 
to B. To realize these conditions, we define the proximity , 
denoted by Pr, of the m th (as A) and the n th (as B) 
motion components in Equation 2 where S mn is the scale 
of the double-ellipse such that the n th motion component 
falls on the boundary of the double-ellipse of the m th 
motion component. Note that if S mn is smaller than 1, 
then the n th motion component lies within the double¬ 
ellipse. Figures 2(a) and 2(b), respectively, show how the 
wedge and double-ellipse are located with respect to the 
m th motion component. 


b 2 





Figure 2. Spatial realization of wedge and double-ellipse are 
illustrated in 2(a) and 2(b). 2(c) shows how we define 0 and ijj. 

Pr(M m , M n ) = | a ^1 < thH ' ^ < the 

( 2 ) 


thep and the are proximity parameters and will be dis¬ 
cussed in detail in the implementation details section. 
Figure 2(c) illustrates how 0 and ip are defined. Given the 
proximity measure in Equation 2, we define the reachability 
set as {(M m , M n )\ Pr(M m ,M n ) < l}. Aiming to mine 
motion patterns, it is disadvantageous to have a motion 
component that cannot reach any other motion component 
(given z, Mj : Pr(M i: Mj) > 1) or no motion component 
is capable of reaching it (given z, Mj : Pr(Mj, Mi) > 1), a 
condition we refer to as a motion component being blocked. 
Such a situation would lead to breaking down motion 
patterns into smaller partitions, preventing us from extract¬ 
ing complete movement behaviors. In order to minimize 
blocked motion components, for each motion component 
Mi , we include in the reachability set the pair ( Mi,Mj ) 
where Pr(Mi,Mj) is minimum with respect to j and 
less than search distance. In the same fashion the pair 
(M z ,Mi) should be included. Finally, it is advantageous 
to be able to detect short range semi-lateral movements. 
This property will allow us to merge similar but parallel 
motion patterns together. In order to accommodate this, 
we include (M m ,M n ) pair in the reachability set if the 
n th motion component lies within a low radius-wide angle 
circular sector (wedge) aligned with respect to m th motion 
component and their flow directions are similar. Formal¬ 
izing the described condition, we include (M m ,M n ) pair 
in reachability set if thfi, \ip\ < th ^ and \p mn \ < th™. 
We should note that in [2 ], [27] or any other method 
that learn the probability density function of the underlying 
dynamics of the entire data, reachability can be defined 
between any two points in the data space by employing 
approaches like MCMC or sequential sampling. On the 
other hand, methods like [36], [26] and this work define 
reachability only between their intermediate representa¬ 
tions. These intermediate representations are the GMM 
components, GP regression model of tracklets and motion 
components in aforementioned works, respectively. Our 
intermediate representations, motion components, are rough 
approximations of GMM components in [36] as we do not 
estimate the covariance matrices of the motion components. 

2.4 Forming Motion Patterns 

We use two concepts of path reachability and signature 
to form motion patterns. The i th motion component is 
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Figure 3. 3(a) shows motion components in red dots and the corresponding arrows refer to their flow direction. Using the double-ellipse 
and wedge conditions, reachable motion components are connected with blue lines respectively in 3(b) and 3(c). Green lines are between 
pairs that happened to be reachable after the unblocking process. Although reachability is asymmetric while the lines are symmetric, the 
asymmetry should be clear from the flow direction of the motion components. 3(d) illustrates all reachable pairs generated via the double¬ 
ellipse and wedge condition after applying the unblocking process. These pairs create the reachability set. Using the reachability set, the 
signature for each motion component is computed. 3(e), 3(f), 3(g) and 3(h) are four examples of the signature for the cyan motion components 
where the components not in the signature are grayed out. Finally, using the weighted Jaccard distance on motion components’ signature, 
we form motion patterns. 3(i) shows five motion patterns (indicated in green, violet, cyan, pink and blue) that our proposed method finds on 
this sample data. 


path reachable from the j th motion component if there 
is a chain of motion components from the j th to the i th 
motion component such that each link is in the reachability 
set. The signature of the i th motion component is the set 
of motion components that are either path reachable from 
the i th motion component or from which the i th motion 
component is path reachable. This is a novel representation 
that provides local and global properties of motion com¬ 
ponents simultaneously and therefore comparison between 
two motion components using their corresponding signa¬ 
tures would reflect both their local similarities and their 
contribution in global behaviors. Mathematically speaking, 
the reachability set can be shown as a directed graph 
while each node represents a motion component. An edge 
exists between two nodes if and only if the pair of motion 
components corresponding to those two nodes exists in the 
reachability set. Given a node, we can obtain the signature 
by applying depth first search on the graph and the reversed 
graph. Finally, the distance between two motion compo¬ 
nents is defined as the weighted Jaccard distance ( WJD ) 
between their associated signatures ( Sig ). The weighting 
is necessary because in practice, some areas have higher 
density of motion components (many motion components 
are located in a small region). This effect will skew our 
results since two motion components might be deemed 
similar by the unweighted Jaccard distance if their paths 
are quite different but they intersect in a region with many 
motion components. To counteract this effect, the Jaccard 
distance is weighted by a factor that assigns low values to 
the motion components that are located in dense regions 
and vice versa. The precise definition for the weighted 
Jaccard distance between the signatures of the m th and 
n th motion components is formalized in Equation 5. We 
extract motion patterns via agglomerative clustering with a 
distance cutoff, single linkage and WJD as distance metric. 


It is worth mentioning that when single linkage is used, 
the results will be the same as clustering via thresholding 
and forming weakly connected components. Despite this, 
our method is different from [ 36 ] as we use the weighted 
Jaccard distance between motion components’ signatures as 
the distance metric. [ 36 ] forms motion patterns by finding 
weakly connected components on a graph that represents 
the reachability between GMM components where only 
local similarity of motion components is considered. There¬ 
fore, it would group globally different motion behaviors to¬ 
gether if they share a common GMM component (note that 
GMM component in [3 ] have actual time information), 
a scenario that occurs in merging or diverging of motion 
patterns. 


WJDl 


Q n (m,n) = {Sig(M m ) n Sig(M n )} 
Qu( m , n ) = {Sig(M m ) U Sig(M n )} 
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( 3 ) 

( 4 ) 

( 5 ) 


Given a sample set of motion components, Figure 3 illus¬ 
trates the steps which our method takes to form motion 
patterns in a trajectory dataset. 


3 Experimental Results 

In this section, we first give a brief description of the 
datasets that were used for evaluation. Then, we show 
experimental results of our proposed trajectory clustering 
method on different datasets and finally compare them 
to the baseline methods. Due to the space limitation, the 
model parameters used to generate outputs associated with 
different datasets are provided in supplemental material. 
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3.1 Datasets 

We used five different datasets to test the proposed method. 
These datasets are among the ones that are used by many 
papers in the literature and vary greatly in their properties, 
such as the number of trajectories, average number of points 
per trajectory, sampling density, spatial separation and 
complexity. Experimental results indicate that our proposed 
method is an effective solution for the trajectory clustering 
task regardless of the dataset properties. 

• Vehicle Motion Trajectory Dataset: This dataset con¬ 
tains 1500 trajectories gathered by tracking vehicles at 
a traffic intersection. These trajectories are annotated 
manually; each trajectory is assigned to one of 15 
trajectory classes. The mean number of points per 
trajectory is 96. This dataset is available at [37]. 

• Atlantic Hurricane Dataset (HURDAT2): This 
dataset is provided by the National Hurricane Ser¬ 
vice (NHS) and contains 1740 trajectories of Atlantic 
Hurricanes from 1851 through 2012, with trajectories 
containing 27 points on average. NHS also provides 
annotations of typical hurricane tracks for each month 
throughout the annual hurricane season that spans 
from June to November. In order to evaluate how close 
the motion patterns mined by our method are to the 
NHS annotations, we divided the Atlantic Hurricane 
Dataset into six subsets, one for each month. Trajecto¬ 
ries that span more than one month were split to ensure 
each month includes only activity occurring within its 
span. This dataset is available at [3£]. 

• Swainson’s Hawks Dataset: This dataset contains 
43 trajectories that trace the migration of Swainson’s 
hawks. A description of the hawks’ migration paths is 
provided in [39], which states that the hawks converge 
on the Gulf of Mexico coast, travel southward follow¬ 
ing a narrow path across the Andes in Colombia, then 
proceed along the east side of the Andes to central 
Argentina, where they spend the austral summer before 
returning north using largely the same route. The 
average number of points per trajectory in this dataset 
is 105. This dataset is available at [40]. 

• The Greek Trucks Dataset: This dataset contains 
1100 trajectories from 50 different trucks delivering 
concrete around Athens, Greece. As expected, the 
trucks follow highways giving the trajectories a dis¬ 
tinctive appearance. The average number of points per 
trajectory is 86. This dataset is available at [41]. 

• The NGSIM Lankershim Dataset: This dataset con¬ 
tains detailed vehicle trajectory data on Lankershim 
Boulevard in the Universal City neighborhood of Los 
Angles, CA on June 16, 2005. The dataset corresponds 
to two 15-minutes periods of 8:30 am to 8:45 am 
and 8:45 am to 9:00 am obtained by five video 
cameras. Lor our experiments, we extracted portions 
of trajectories captured by camera NO. 2 in 8:30 am to 
8:45 am period as it is covering the busiest intersection 
in the dataset. This subset contains 1095 trajectories 
and the average number of points per trajectory is 305. 
The full Lankershim dataset and its annotations are 
available at [42]. 


3.2 Pre-Processing 

Lor all experiments, data was first normalized so that all 
the trajectories were in the bounding box determined by 
x G [0,1000] and y G [0,1000]. The Atlantic Hurricane 
Dataset went through further pre-processing: after splitting 
the dataset by months, each of the resulting subsets was 
pruned by removing trajectories consisting solely of a single 
coordinate pair repeated one or more times. 

3.3 Intrepreting Output 

Before we discuss experimental results, we first briefly 
explain how to read and interpret the output of the proposed 
algorithm. Each discovered motion pattern is displayed 
on top of plotted trajectories. To avoid overcrowding the 
figures, only a random subset of trajectories are visualized. 
The color of the motion pattern denotes the direction of 
the motion. A color wheel that appears in each figure 
serves as the legend for translating color into direction of 
motion. Lastly, we must introduce the reasoning behind 
our handling of merging and diverging trajectories. An 
example of such a scenario is illustrated in Ligure 4(a), 
where two clusters of trajectories partially overlap. Our 
proposed method recovers three motion patterns in this 
case: two separate motion patterns for the distinct portions 
of the two clusters and a separate motion pattern for their 
merged portion. Therefore, it must be noted that a single 
trajectory may pass through several motion patterns. Using 
our proposed method, trajectories shown in Ligure 4(a) 
produce motion patterns shown in Ligures 4(b), 4(c) and 
4(d). 



(a) (b) (c) (d) (e) 


Figure 4. An example for merging in 4(a) where the proposed 
method has found two separate motion patterns of 4(b) and 4(d) in 
addition to the common part 4(c). In 4(a), colors of black and blue 
are used only to distinguish two classes of trajectories according to 
the annotations. Colorwheel is shown in 4(e). 

3.4 Evaluations on Vehicle Motion Trajectory 
Dataset 

Examining the outputs of the proposed method versus 
annotations shown in Ligure 5(a), we see that the pro¬ 
posed algorithm recovers the motion patterns present in 
the dataset, with a few differences from the annotated 
version. Lirst of all, the annotations categorize the traffic 
in each of the parallel lanes in Ligure 5(b) into separate 
clusters. Because the motion in the left group of three 
lanes is very similar, the proposed algorithm recovers it as 
a single motion pattern. The same applies to the three-lane 
group on the right. Second, for reasons stated above, the 
turning trajectories shown in Ligure 5(c) are segmented to 
differentiate their distinct portions. Ligures 5(d) and 5(e) 
correspond to the two directions of traffic on the main 
highways. Ligure 5(f) corresponds to one of the left turns 
and is especially large since it contains the bulk of the left¬ 
side outgoing traffic. Ligures 5(g), 5(h), and 5(i) correspond 
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Figure 5. 1 5 different annotated classes of trajectories in Vehicle Motion Trajectory Dataset are shown in 5(a) with different colors. To have 
a more clear visualization, we illustrate classes belong to main highways in 5(b) and the rest in 5(c). Extracted motion patterns from Vehicle 
Motion Trajectory Dataset are illustrated in 5(d) to 5(o). Colorwheel is shown in 5(p). 


to the incoming traffic from the left side. This is an example 
of the diverging behavior where the motion pattern from 
Figure 5(h) diverges into the motion pattern turning right in 
Figure 5(i) and the motion pattern going straight in Figure 
5(g). Motion patterns in Figures 5(j) and 5(m) reflect a U- 
turn, where the former is a portion of the U-turn that merges 
with the traffic shown in Figure 5(e) and the latter is the 
distinct part of the U-turn. Figures 5(k) and 5(1) show two 
different access roads. Figure 5(n) shows a right turn and 
Figure 5(o) shows a rarely taken left turn. 

3.5 Evaluations on Atlantic Hurricane Dataset 

We evaluated the performance of the proposed method on 
the Atlantic Hurricane Dataset by comparing its output 
with the annotations provided by the National Hurricane 
Service (NHS). Prevailing hurricane tracks for each month 
are indicated by white arrows in Figures 6(a), 6(d), 6(i), 
6(m), 6(s) and 6(v). Examining the output for each month, 
we see that the proposed algorithm recovers motion patterns 
closely resembling the prevailing hurricane tracks for June 
in 6(b) and 6(c), for October in 6(t) and 6(u), and for 
November in 6(w). For July, 6(e) reflects the rightmost 
track, while 6(f) reflects the shared path of the leftmost 
tracks, and 6(g) and 6(h) serve as the distinct portions of 
the two leftmost paths. For August, 6(j) reflects the bottom 
left prevailing track, while 6(k) and 6(1) correspond to 
the splitting tracks on the top right. September hurricane 
tracks in 6(m) are split into two groups of three. The 
left group of three is output as motion patterns 6(o), 6(r), 
6(p), while the right group of three is output as 6(n) for 
the arrow pointing northwest and 6(q) for the two arrows 
turning towards northeast. There is a considerable variation 
in trajectory density across different months in the dataset. 
As a general rule, months with fewer hurricanes, such as 
June, July and November, require more relaxed reachability 
conditions (larger double-ellipse and wedge radius), while 
months with high number of hurricanes, such as August, 
September and October, require stricter ones. 

3.6 Evaluation on Swainson’s Hawks Dataset 

We evaluated the proposed method’s performance on the 
Swainson’s Hawks Dataset by comparing the algorithm’s 
output with the study summary provided in [3S]. In par¬ 
ticular, we have been able to retrace the southward and 


northward migration paths of the birds in Figures 7(a) and 
7(b) respectively. 

3.7 Evaluation on Greek Trucks Dataset 

Given the dataset, we find the major highways that are 
taken by trucks. As previously noted, in the case of two 
routes merging, the algorithm will find three patterns: two 
for the routes before they merge, and one for the combined 
routes after they merge. The patterns in Figures 8(b) and 
8(c) merge into the pattern in Figure 8(a). Similarly, the 
pattern in Figure 8(d) diverges into the patterns shown in 
Figures 8(g) and 8(h). Finally, the patterns shown in Figures 
8(f), 8(i), and 8(j) merge to create pattern shown in Figure 
8(e). 

3.8 Evaluation on NGSIM Lankershim Dataset 

The traffic lanes traveling upward are reflected in 9(a), 9(b) 
and 9(c). The three downward lanes are shown in 9(d), 
9(e) and 9(f). Figures 9(g), 9(h) and 9(j) reflect the left 
turns, while 9(i) and 9(k) reflect right turns. Traffic crossing 
Lankershim boulevard is reflected in 9(1), 9(m) and 9(n). 

3.9 Experimental Comparison 

Various quantitative measures for evaluating the quality of 
clusters exist in the literature. [43] , [44], [3] and [45] 
computed Correct Clustering Rate (CCR) as a measure to 
evaluate clustering results against the ground truth or man¬ 
ual annotations. [1] used an information theoretic criterion 
proposed by Meila [46], Variation of Information (VI), to 
validate obtained clusters. VI determines the amount of 
information lost and gained between two different clus¬ 
terings of data. Masciari compared inter-class and intra¬ 
class similarity of clusters in [47], [48]. All of these and 
similar measures, however, were used to evaluate clusters 
of entire trajectories. Since this does not align with the 
goals of our work, an attempt to use the aforementioned 
metrics would not yield a meaningful comparison. Rather 
than clustering entire trajectories, the proposed algorithm 
mines regional trends in trajectory data like [49], [5' ], 
[1 ], [15], [16] and [17]. [4 ( ] evaluates obtained results 
by visual comparison with GSP [51] and PrefixSpan [52]. 
[f ] offers a visual comparison of results with TRACLUS 
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(a) (b) 



(c) 




(i) 




c 

(V) (w) (X) 

Figure 6. Extracted motion patterns from Atlantic Hurricane 
Dataset. Colorwheel is shown in 6(x). 






[] ]. TRACLUS authors, in turn, state that there is no well- 
defined measure for density-based clustering methods, and 
suggest using a metric consisting of sum of squared error 
and noise penalty to, in authors’ own words, ’’get a hint of 
the clustering quality”. This metric cannot be easily adapted 
to methods other than variations of TRACLUS. Ferreira 
et.al. [14], who use the subset of the Atlantic Hurricane 
Dataset for their evaluations, report that their results are 
visually consistent with expected hurricane behavior, as 
does [5 ]. [1 ] provides a visual analysis and suggests a 



Figure 7. Extracted motion patterns from Swainson’s Hawks 
Dataset. Colorwheel is shown in 7(c). 



(a) (b) (c) 



(d) (e) (f) 


V 

(g) (h) (i) 

^' c 

CD (k) 


Figure 8. Extracted motion patterns from The Greek Trucks 
Dataset. Colorwheel is shown in 8(k). 


semantic interpretation of their results on the Greek Trucks 
Dataset. Staying close to the spirit of the works that mine 
regional trends, we will demonstrate the effectiveness of the 
proposed method by visually comparing our results with 
those of TRACLUS [18], DivCluST [15], [17] and [16]. 
Due to the space limitation in the paper, we were not able 
to provide all the visualizations for each of the baseline 
methods. Instead, we illustrated cases where they could not 
handle the difficulties provided by the datasets. Compared 
to the baseline methods, our algorithm can successfully 
face those challenges. The goal of TRACLUS [18] is to 
detect similar portions of trajectories, which semantically 
correspond to the output of our proposed method. More 
formally, given a set of trajectories, TRACLUS partitions 
trajectories, clusters them, and outputs a representative 
trajectory for each cluster. We used TRACLUS imple¬ 
mentation available at [54]. It contains a program that 
estimates optimum parameter values. According to [1 ], 
these values are approximate and may differ slightly from 
the true optimal values. Therefore, we ran TRACLUS on 
each dataset multiple times, tuning each of the parameters. 
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(k) (1) (m) (n) (o) 

Figure 9. Extracted motion patterns from the NGSIM Lankershim 
Dataset. Colorwheel is shown in 9(o). 


This is not unlike the approach to parameter selection that 
TRACLUS authors take. Lee et.al. [1 ] suggest a heuristic 
approach to parameter selection, where optimal 5 is selected 
to minimize the entropy of obtained clusters. Further, the 
quality of clusters is estimated using the sum of squared 
error and the noise penalty. Finally, [18] points out that the 
final selection of optimal parameters for TRACLUS is made 
using visual inspection and domain knowledge. One of the 
challenges that TRACLUS faces, as [5( ] also mentions, 
is that it is not capable of simultaneously extracting both 
dense and sparse trajectory clusters because its parameters 
are optimized globally. Modifying its parameters such that 
it finds sparser clusters leads to redundant clusters in denser 
regions. Figure 10(a) shows clusters found by TRACLUS in 
Vehicle Motion Trajectory, when optimized £ and MinLns 
are used. We tuned these parameters to allow TRACLUS 
to extract clusters in both sparse and dense regions, but 
it results in many short, local clusters and redundant ones 
shown in Figure 10(b). Our proposed method can handle 
variation in density as is illustrated in Figure 5. 



(a) (b) 

Figure 10. TRACLUS parameters (e,MinLns) used in 10(a) and 
10(b) are respectively as follows: (20,12), (20,3). 


DivCluST [15] is an algorithm that seeks to find re¬ 
gional typical moving styles in the form of mean lines. 
Performance of DivCluST on the Greek Trucks Dataset 
is shown in Figure 11(a) where parameters are optimized 
according to the method described in the paper. Each arrow 
represents a mean line where the thickness refers to the 
frequency of that style and the color corresponds to the 
speed of that style. Warmer, reddish colors are for faster 
styles while cooler, bluish colors are for slower styles. 
This algorithm has trouble when there is large variation 
in the trajectory density. The Greek Trucks Dataset has 
this property which causes problems. In the high density 
regions, the mean lines are very cluttered and overlapping. 
Due to the Kmeans-type model, there are very similar mean 
lines. This is because if the random initial clusters are close 
together in a high density region, a Kmeans-type algorithm 
will often keep them close together. In the low density 
regions, there are mean lines that are not representative, 
such as the mean lines within the yellow box in Figure 
11(b). This is because in low density regions, quite different 
representative segments are clustered together, producing 
mean lines that are dissimilar from all the representative 
segments. Additionally, because the model is restricted to 
straight mean lines rather than curves, motion that would 
be better described as a curve is instead required to be 
described as one long mean line or a sequence of short 
mean lines. An example of this are the long cyan arrows 
within the purple box in Figure 11(b). Our algorithm deals 
much better with variation in density and has the ability to 
find curved motion patterns. 



(a) 



Figure 11 . DivCluST on the Greek Trucks Dataset with parame¬ 
ters thi en = 0.17, th spd = 0.50, and £;=95 is shown in 11(a). Yellow box 
in 11 (b) indicates a region with low trajectory density while the purple 
box contains an example of curvy motion patterns that are modeled 
as straight lines by DivCluST. 
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Giannotti et.al. [17] proposed an algorithm that seeks 
to find aggregate motion behaviors from trajectories. These 
behaviors are defined as sequences of rectangular regions. 
In Figure 12, some of the aggregated motion behaviors 
are illustrated where each is represented as a sequence of 
rectangular black regions. The order of the sequence is 
shown by the black arrows. First, it is important to note 
that this algorithm gives very redundant results. Many of 
the motion behaviors are very similar and some are even 
subsets of each other. This requires digging through many 
patterns to find the distinguishable ones. Although there 
are 80 generated patterns in total for the Vehicle Motion 
Trajectory Dataset, only 6 representative ones are shown in 
Figure 12. In other words, of the patterns not shown, each 
of them has the same shape or is a subset of the shown 
patterns. Requiring regions to be rectangular restricts the 
shape of extracted patterns. For instance, in Figure 12(a), 
the upper right part of the larger box covers a lane of 
traffic that should not be included in the pattern. Since 
the rectangular regions are built only based on density 
of trajectories without considering motion properties, they 
are not always suitable to represent motion behaviors. For 
instance, in Figures 12(b), 12(c), and 12(d), the rightmost 
box includes a different lane of traffic in the upper portion 
of the box. Finally, multiple traffic behaviors are sometimes 
combined into a single pattern. For instance, Figure 12(a) 
shows the traffic turning right and going straight combined, 
Figure 12(c) shows the traffic going straight and making 
U-turns combined, Figure 12(e) shows the access road and 
incoming traffic from the left combined and Figure 12(f) 
shows the U-turn and straight traffic combined. Unlike this 
method, the approach which we have proposed in this paper 
does not generate redundant or overlapping motion patterns. 
Our motion components are more flexible than rectangular 
regions used in [17] and our method does not mix multiple 
patterns of traffic in a motion pattern as can be seen in 
Figure 5. 



(d) (e) (f) 

Figure 12. Parameters that [17] has used for Vehicle Motion 
Trajectory Dataset are as: £ min =0.17, <5=0.17, r=100 and £=25. 

Ulm et.al. [16] proposed an algorithm that seeks to find 
clusters in the form of vector fields defined on a connected 
spatial set. While this algorithm performs well on datasets 
that are well structured such as Vehicle Motion Trajectory 


dataset, it is not well suited for those that lack this property. 
This is partially due to the fact that it behaves much like 
the algorithms that cluster trajectories as a whole. It can 
be seen in Figure 13(d) that the vector fields can include 
directions going two opposite ways on a road. This can 
be avoided by changing the weights, but this results in 
vector fields that point perpendicular to the roads. This 
issue is not present in our algorithm since it can find 
two spatially overlapping motion patterns with different 
directions. Secondly, the clusters are not homogeneous in 
movement. Some of the clusters are very sprawling while 
others have many small, noisy portions. This is not the 
effect of using too few clusters since there is a large amount 
of overlap among them. Instead, this is the effect of using 
whole trajectories to create vector fields. 




(C) (d) 

Figure 13. [16] is evaluated on the Greek Trucks Dataset with the 
cluster threshold of 110, the prototype threshold of 100, the position 
weight of 1 and the direction weight of 100. All other parameters 
were set to the default values. Figures 13(a), 13(b), 13(c) and 13(d) 
are the clusters that [16] has found. 

4 Implementation Details 

In this section, the parameters will be discussed. This 
algorithm includes 16 parameters in total that influence the 
output of the method. The intuitive interpretation of each 
of the parameters will be discussed and an approach for 
fine-tuning them in independent groups will be described. 

The first parameter is the number of clusters K , which 
determines the number of motion components. It is impor¬ 
tant for the total number of motion components to be large 
in order to have a good resolution for the patterns. However, 
if the number is too large, some of the motion components 
may end up with only a few flow vectors. These motion 
components may be dramatically affected by noise and 
in some cases even form false motion patterns. It must 
also be noted that clustering is a computationally intensive 
task, hence the selection of K will also be influenced by 
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Figure 14. A subset of NGSIM Lankershim Dataset is used for 
visualizing the effects of parameters tuning. The effects of extremely 
large (3, double-ellipse parameters and so large th e ^ are illustrated 
in 14(a), 14(b) and 14(c), respectively. Figures 14(d), 14(e) and 14(f) 
show the effect of using too small th e Finally, the effect of too large 
th™ is shown in 14(g). 



(a) (b) 

Figure 15. 1 5(a) shows motion components that form the sig¬ 

nature for a motion component depicted in black. This signature is 
computed without using the wedge. Figure 15(b) shows the signature 
for the same motion component that is computed with the wedge 
parameters set to th^ = 120, th™ = 25, th% = 15. 


the desired running time. Generally, datasets that are not 
expected to contain much noise will allow for arbitrarily 
large values of K. 

The second parameter, (3, is also used in clustering. 
This parameter controls how much the velocity of the flow 
vector affects the formation of motion components. If (3 
is too small, the spatial proximity will be the primary 
factor affecting the formation of the motion components, 
and each motion component’s flow will be the average flow 
of its constituent flow vectors. Whenever this is the case, 
the algorithm will likely miss overlapping motion patterns 
with different flow directions, such as those seen in the 
Swainson’s Hawks dataset. The larger (3 becomes, the larger 
role the flow direction plays in clustering. A large (3 will 
make the algorithm sensitive to smaller deviations in flow 
direction, as well as noise. Extremely large (3 will cause the 
flow direction to bias the clustering process, forming mo¬ 
tion components from vectors with similar flow directions 
and dissimilar spatial coordinates, as illustrated in 14(a), 
where /? is set to 1000 , which is much higher than (3 equal 
to 45, used for other results. In general, f3 should be as small 
as possible while still allowing any meaningful overlap 
of patterns to exist. These two parameters completely 
determine the formation of motion components. Thus, these 
parameters can be fine-tuned without running the entire 
model but only visualizing the motion components. This 
will make the parameter selection process far simpler. 

The next parameters are a±, &i, a<i, and 62 - These param¬ 


eters determine the shape and size of the double-ellipse. In 
general, the double-ellipse should be small enough to avoid 
denoting motion components as reachable that belong to 
different motion patterns, yet large enough to jump gaps in 
the motion component distribution within motion patterns. 
Example in 14(b) illustrates the outcome of setting the ai 
and 62 of the ellipses too high. The ’’wider” ellipses jump 
the gap between lanes. Similar logic applies to manipulating 
< 22 , bi values. The parameter the^ should be chosen in a 
similar way except that this parameter deals with the flow 
direction rather than spatial position, the^ should be large 
enough to capture deviations within motion patterns but 
not so large to make motion components reachable which 
belong to different motion patterns, as is demonstrated in 
14(c), where the^ was set to 120 from its usual 12. Too low 
value, on the other hand, will yield results where even small 
differences in flow direction will make motion components 
unreachable. This effect is demonstrated in Figures 14(d), 
14(f) and 14(e), where the right turn is spit into three motion 
patterns. 

The parameter a also deals with the flow direction of 
motion components. If this parameter is high, the algorithm 
expects well-formed curves, and if it is small, the algorithm 
will look for similar flow directions between reachable 
motion components rather than well-formed curves. If this 
parameter is set too high, the algorithm will be sensitive 
to noisy data and find curves where there are none. On 
the other hand, if it is set too low, the algorithm will have 
trouble finding curves - leading to the same problem as 
the one demonstrated in 14(d), 14(f) and 14(e). Because 
the algorithm expects curves, it sometimes connects two 
motion components with very different flow directions. To 
avoid this situation, a threshold the is placed on the absolute 
value of 0. In general, this will be two or three times the 
value of the 

The parameters so far discussed determine the core of the 
reachability relation. Among the reachability parameters, 
search distance deals with the unblocking process and 
the wedge parameters allow semi-lateral motions. To test 
this core reachability, the path reachable motion compo¬ 
nents and signature for some motion components can be 
computed with discounting the semi-lateral motion and 
unblocking process by setting the wedge parameters to 0 
and search distance to 1. Then the signature for a motion 
component can be plotted and evaluated. However, for 
some datasets, these signatures will be proper for most 
motion components, but for other motion components, 
the signatures will unexpectedly end because of blocked 
motion components. To fix this, the search distance can 
be increased. However, if search distance is too large, 
the algorithm will make unreasonably far away motion 
components reachable. Once again, search distance value 
can be evaluated by visualizing the signatures of a sample 
of the motion components. 

Additionally, the wedge parameters can be set to nonzero 
values to include semi-lateral reachability and improve the 
signatures. In some datasets, the signatures will not cover 
the entire width of motion patterns, as in Figure 15(a), 
where a signature for a motion component is plotted. In 
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these cases, the wedge should be used to improve the 
results, thg should be just high enough to include the width 
of motion patterns with somewhat noisy flow direction, th™ 
should be just high enough to include gaps within the width 
of the motion pattern, th^ should be high enough to expand 
a signature to the width of the motion pattern within a 
short distance, as demonstrated in Figure 15(b). An example 
of wedge parameters at work is given in figure 14(g), 
where th ^ = 90, th™ = 20, thfi = 15, and the algorithm 
recovers all three parallel southbound lanes as one motion 
pattern. This concludes the reachability parameters that 
can be fine-tuned and tested independently by looking at 
the signatures of some of the motion components. The 
impact of wedge parameters can be examined by visualizing 
signatures. Finally, there is one parameter for the clustering 
of motion components. The cutoff value, determines the 
size of the motion patterns. The larger the cutoff, the larger 
the motion patterns will be. 

5 Discussion 

In many applications, the task of automatically detecting 
changes in trends is just as interesting as uncovering exist¬ 
ing ones. For example, in traffic control, sudden disappear¬ 
ances of motion patterns may indicate lane or street closures 
or blockages due to accidents, fallen trees, flooding, and 
other causes. Changes in motion patterns of animals may 
be indicative of changes in their environment, such as those 
due to urban expansion or pollution. These changes are of 
particular interest to conservation biologists. Furthermore, 
zoologists are frequently interested in analyzing seasonal 
changes in movement of animals. In the domain of com¬ 
merce, merchandisers and marketing professionals can use 
changes in shopper traffic patterns to improve advertisement 
and product visibility. We can mine changes in trends of 
trajectory data by either discovering newly emerged motion 
patterns or by finding motion patterns that no longer occur. 
Given trajectories observed from time t\ to t- h and from 
time ti +1 to tj, we generate two sets of motion patterns, 
G and G' respectively, using our proposed algorithm. G 

contains motion patterns gi where i = 1, 2,_, n and G' 

contains motion patterns g[ where i = 1, 2,..., n'. Then, 
for any g[, using Kullback-Leibler (KL) divergence, we 
try to find a similar motion pattern in G. If no match 
is found, it means that g[ is a newly emerged motion 
pattern, a trend that was not previously observed in t\ to U 
interval. Repeating the process for all gi , we can detect the 
disappearance of a motion pattern if no match is found in 
G'. In order to be able to use KL divergence for comparing 
two motion patterns, we need to have their probability 
distribution functions. The pdf of gi and g[ denoted as p 9i 
and p g / respectively, can be obtained by learning mixtures 
of Gaussians from flow vectors that are contained by their 
constituent motion components. Then we draw a sufficient 
number of samples from p g . and evaluate probability of 
their occurrence in p g r. A high probability means that 
two motion patterns are similar as their KL divergence 
is low. [36] has employed this approach to merge large 
numbers of similar motion patterns that occur at different 
times together while [55] performs event classification by 


matching the distribution of motion patterns that minimizes 
KL divergence. 

6 Conclusion 

In this paper, we considered the general task of trajectory 
clustering using a novel approach inspired by the motion 
pattern idea. Our method consists of four main steps. 
First, we break down trajectories into flow vectors and 
then, using Kmeans clustering, we extract motion com¬ 
ponents. In the third step, we use the double-ellipse and 
the wedge conditions in addition to unblocking procedure 
to find reachable pairs of motion components. Finally, 
using the path reachability and signature concepts, we 
form motion patterns via agglomerative clustering with the 
weighted Jaccard distance between motion components’ 
signatures. We evaluated our proposed method on five 
different datasets. Experimental results indicate that our 
motion pattern approach gives an effective solution to 
the general task of trajectory clustering regardless of the 
dataset properties. Extracted motion patterns closely fit the 
annotations, prevailing paths, or descriptions of trajectory 
datasets where available. We comprehensively discussed 
the effects of model parameters and provided a selection 
process for them. Also, we noted that the actual optimum 
set of parameters will rely on domain knowledge as well as 
specific analytical goals. In addition, we discussed how our 
proposed model is well suited for automatically detecting 
changes in frequent behaviors of trajectories over time. 

Overall, we believe that we have provided a new ap¬ 
proach for understanding trajectory behavior. Its output 
is comparable to the output of those trajectory clustering 
methods that look for regional similarities among trajec¬ 
tories. It is capable of handling a variety of challenges 
provided by different datasets. Our proposed method can 
provide data analysts a good starting point for under¬ 
standing the hidden behavioral patterns in enormous and 
complex trajectory datasets. 
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